\chapter{Advanced Topics}\label{ch:advanced-topics}

[{\em Alas, this chapter is only partially written.  Contact the authors for
assistance with any of these topics.}]
$$
$$

\section{Lazy Evaluation: Working with Views}\label{sec:views}

In the Vision Workbench, the \verb#ImageView<># class gives you access
to raw, reference-counted pixel data stored in RAM.  However, this is
just one instance of a more general {\em View} concept that is central
to the inner workings of the Vision Workbench library.  In this
section, we will discuss the view concept in detail and show how you
can leverage the full flexibility of VW by creating and optimizing
your own {\em Views}.

\subsection {The View Concept}

Consider what could happen when you chain three image processing
operation together.

\begin{verbatim}
    result = image1 + image2 + image3;
    result = transpose( crop( image, x, y, 31, 31 ) );
\end{verbatim}

In order to perform this computation, the C++ compiler introduces a
temporary image like this:

\begin{verbatim}
    ImageView<PixelGray<uint8> > tmp = image1 + image2;
    result = tmp + image3;
\end{verbatim}

And this:

\begin{verbatim}
    ImageView<PixelGray<uint8> > tmp = crop( image, x, y, 31, 31 );
    result = transpose(tmp);
\end{verbatim}

For simple operations such as addition, the introduction of a
temporary image like this can be terribly inefficient because it
causes a large, image-sized block of memory to be allocated and
deallocated, and it requires a second pass over the data.  

It is more efficient to allocate a single temporary
\verb#PixelGray<uint8>#; using it as you iterate over the image
computing the result one pixel at a time.  This is how a hand-coded
implementation would be written by a cognizant programmer.

To avoid such inefficiencies, the Vision Workbench resorts to {\em
  lazy evaluation}; postponing operations on {\em Views} until the
last possible moment before the result is needed.  For example,
applying the addition operator to a pair of images does not
immediately compute their sum.  Instead, {\em the + operator returns
  an image sum object.}  This object is a {\em View}: an object that
behave just like an image, but represents a processed view of the
underlying data (see Table \ref{tbl:view-concept}).  The actual
computation of the sum is postponed until later when it is needed; for
example when the user assigns it to an \verb#ImageView<>#, thereby
causing the sum to be computed and stored in memory.  This process of
finally evaluating one view into another is called {\em
  rasterization}.
  \begin{table}[tb]\begin{centering}
 \begin{tabular}{|c|l|} \hline
   Getting dimensions &   \verb#img.cols()#   \\
                      &   \verb#img.rows()#   \\
                      &   \verb#img.planes()# \\
   \hline
   Accessing pixels  & \verb#img(col,row)# \\
                     & \verb#img(col,row,plane)# \\
   \hline
   STL iterator      & \verb#ImageViewBase<...>::iterator# \\
                     & \verb#img.begin()# \\
                     & \verb#img.end()#\\
   \hline
   Pixel iterator   & \verb#ImageViewBase<...>::pixel_iterator# \\
                    & \verb#img.origin()# \\
   \hline
 \end{tabular}
 \caption{In the Vision Workbench, a {\em View} is any object that supplies the above methods and inherits from ImageViewBase.}
 \label{tbl:view-concept}
 \end{centering}\end{table}

Nested function calls produce nested View types, e.g. a sum object
containing a sum object plus an image, and so on.  In this fashion, a
series of image processing operations builds up a {\em View Tree} that
can represent an extensive chain of operations.  The tree of
operations are collapsed into the inner loop when they are rasterized,
allowing the compiler to efficiently allocate small temporaries.
Rasterization is also an opportunity to make decisions about how to
optimize the image processing operations.  For example, it may
actually be {\em more} efficient in some cases to introduce an entire
temporary image in between two operations.  A convolution operation
followed by another convolution operation is much more efficient when
implemented in this manner.  The Vision Workbench rasterization engine
does its best to make efficient decisions about when and how to
introduce temporaries when rasterizing a view tree.  We'll have more
to say on this subject in the sections below.







\begin{verbatim}
   /// An image view for performing image correlation
  template <class ImageT, class PreProcFuncT>
  class CorrelatorView : public ImageViewBase<CorrelatorView<ImageT, PreProcFuncT> > {

    typedef typename ImageT::pixel_type pixel_type

    ImageViewRef<pixel_type> m_left_image, m_right_image;
    PreProcFuncT m_preproc_func;

    // Settings
    BBox2i m_search_range;
    Vector2i m_kernel_size;

    public:
      typedef typename ImageT::pixel_type pixel_type;
      typedef pixel_type result_type;
      typedef ProceduralPixelAccessor<CorrelatorView> pixel_accessor;
      
    CorrelatorView(ImageViewBase<ImageT> const& left_image, ImageViewBase<ImageT> const& right_image, 
                   PreProcFuncT const& preproc_func) :
      m_left_image(left_image.impl()), m_right_image(right_image.impl()), 
      m_preproc_func(preproc_func) {

      // Set some sensible default values
      m_search_range = BBox2i(-50,-50,100,100);
      m_kernel_size = Vector2i(24,24);
    }

    // Standard ImageView interface methods
    inline int32 cols() const { return m_left_image.cols(); }
    inline int32 rows() const { return m_left_image.rows(); }
    inline int32 planes() const { return 1; }
    
    inline pixel_accessor origin() const { return pixel_accessor( *this, 0, 0 ); }
    
    inline pixel_type operator()(double i, double j, int32 p = 0) const {
      vw_throw(NoImplErr() << "CorrelatorView::operator()(double i, double j, int32 p) has not been implemented.");
      return pixel_type(); // Never reached
    }

    /// \cond INTERNAL
    typedef CropView<ImageView<pixel_type> > prerasterize_type;
    inline prerasterize_type prerasterize(BBox2i bbox) const {

      // The area in the right image that we'll be searching is
      // determined by the bbox of the left image plus the search
      // range.
      BBox2i left_crop_bbox(bbox);
      BBox2i right_crop_bbox(bbox.min() + m_search_range.min(),
                             bbox.max() + m_search_range.max());
      
      // The correlator requires the images to be the same size. The
      // search bbox will always be larger than the given left image
      // bbox, so we just make the left bbox the same size as the
      // right bbox.
      left_crop_bbox.max() = left_crop_bbox.min() + Vector2i(right_crop_bbox.width(), right_crop_bbox.height());

      // Finally, we must adjust both bounding boxes to account for
      // the size of the kernel itself.
      right_crop_bbox.min() -= Vector2i(m_kernel_size[0], m_kernel_size[1]);
      right_crop_bbox.max() += Vector2i(m_kernel_size[0], m_kernel_size[1]);
      left_crop_bbox.min() -= Vector2i(m_kernel_size[0], m_kernel_size[1]);
      left_crop_bbox.max() += Vector2i(m_kernel_size[0], m_kernel_size[1]);

      // Log some helpful debugging info
      vw_out(DebugMessage, "stereo") << "\t   search_range: " << m_search_range << std::endl;
      vw_out(DebugMessage, "stereo") << "\t left_crop_bbox: " << left_crop_bbox << std::endl;
      vw_out(DebugMessage, "stereo") << "\tright_crop_bbox: " << right_crop_bbox << std::endl;

      // We crop the images to the expanded bounding box and edge
      // extend in case the new bbox extends past the image bounds.
      ImageView<ImagePixelT> cropped_left_image = crop(edge_extend(m_left_image, ZeroEdgeExtension()), left_crop_bbox);
      ImageView<ImagePixelT> cropped_right_image = crop(edge_extend(m_right_image, ZeroEdgeExtension()), right_crop_bbox);
      ImageView<MaskPixelT> cropped_left_mask = crop(edge_extend(m_left_mask, ZeroEdgeExtension()), left_crop_bbox);
      ImageView<MaskPixelT> cropped_right_mask = crop(edge_extend(m_right_mask, ZeroEdgeExtension()), right_crop_bbox);
      
      // We have all of the settings adjusted.  Now we just have to
      // run the correlator.
      vw::stereo::PyramidCorrelator correlator(BBox2(0,0,m_search_range.width(),m_search_range.height()),
                                               Vector2i(m_kernel_size[0], m_kernel_size[1]),
                                               m_cross_corr_threshold, m_corr_score_threshold,
                                               m_cost_blur, m_correlator_type);

      // For debugging: this saves the disparity map at various pyramid levels to disk.
      if (m_debug_prefix.size() != 0) {
        std::ostringstream ostr;
        ostr << "-" << bbox.min().x() << "-" << bbox.max().x() << "_" << bbox.min().y() << "-" << bbox.max().y() << "-";
        correlator.set_debug_mode(m_debug_prefix + ostr.str());
      }
      
      ImageView<pixel_type> disparity_map = correlator(cropped_left_image, cropped_right_image, 
                                                       cropped_left_mask, cropped_right_mask,
                                                       m_preproc_func);
      
      // Adjust the disparities to be relative to the uncropped
      // image pixel locations
      for (int v = 0; v < disparity_map.rows(); ++v)
        for (int u = 0; u < disparity_map.cols(); ++u)
          if (!disparity_map(u,v).missing())  {
            disparity_map(u,v).h() += m_search_range.min().x();
            disparity_map(u,v).v() += m_search_range.min().y();
          }

      // This may seem confusing, but we must crop here so that the
      // good pixel data is placed into the coordinates specified by
      // the bbox.  This allows rasterize to touch those pixels
      // using the coordinates inside the bbox.  The pixels outside
      // those coordinates are invalid, and they never get accessed.
      return CropView<ImageView<pixel_type> > (disparity_map, BBox2i(m_kernel_size[0]-bbox.min().x(), 
                                                                     m_kernel_size[1]-bbox.min().y(), 
                                                                     bbox.width(), bbox.height()));
    }
    
    template <class DestT> inline void rasterize(DestT const& dest, BBox2i bbox) const {
      vw::rasterize(prerasterize(bbox), dest, bbox);
    }
    /// \endcond
  };
\end {verbatim}


\section{Working with Shallow Views}\label{sec:advanced.shallow}

\section{Efficient Algorithms and {\tt pixel\char`\_accessor}}

\section{Rasterization, Efficiency, and Tiled Computation}

\section{Generic Image Buffers}\label{sec:advanced.generic}

\section{The File I/O System}\label{sec:advanced.fileio}

\section{Frequency-Domain Image Processing}\label{sec:advanced.frequency}
